On Credibility, Clarity, and Compliance

Since its inception, MCP has recognized the promise of proteomics and its underlying technologies to significantly move the boundaries of knowledge in biology and medicine. As has been noted (1), proteomics represented a paradigm shift in how experiments were designed and executed and how the resultant data were interpreted and stored. But the journal has also recognized that the value of the proteomic approach and the data generated were only as good as the quality and reliability of that data—a corollary of the computational maxim, “garbage in, garbage out.” Thus, in its very earliest stages, the editorial staff of MCP, particularly Steve Carr, Ruedi Aebersold, and Al Burlingame, began earnest discussions about how a journal should evaluate data arising from large scale, often called “shotgun,” mass spectrometric experiments, which ultimately lead to the first set of guidelines (2). Initially, these questions focused on correctly identifying peptide sequences (as surrogates for the proteins they were derived from) but quickly expanded to the identification and localization of post-translational modifications (PTMs) and mass-spectrometric-based quantification as well (3, 4). In the main, it was decided that manuscripts submitted to the journal should provide sufficient information to allow an appropriate assessment by first reviewers and then readers. At first, the requested information consisted of the methods used for collection and interpretation (basically search parameters) of the data as well as the identifications themselves. As the guidelines evolved, tutorials and checklists were introduced to aid the submission process. The latter were eventually expanded to include checklists for papers of clinical relevance and those reporting glycomic analyses (5, 6). Finally, after first introducing the guidelines simply as recommendations, the journal began enforcing them by subjecting all submitted articles to a review to determine if they were compliant. The compliance check is not, and was never meant to be, a part of the evaluative peer review process, as these checks only determine whether the necessary information has been supplied and do not assess in any way its quality or interpretation. One of the most challenging aspects of the documentation process as it developed was the requirement that authors provide “annotated” spectra for all MS/MS protein identifications based on only a single unique peptide (including the spectra used in peptide mapping fingerprints (PMFs)) and all PTM assignments. Annotated spectra means the labeling of the m/z for all significant peaks in the spectra as well as their fragment ion designations (e.g. y, b, etc. if spectra are from an MS/MS experiment) relative to the sequence being reported. Given the variations in software supporting different instruments and in the outputs of the many individual search engines, this single requirement became an increasingly difficult one to manage. In keeping with the historical “maintenance of the public record” role, which basically requires journals to report all the data that are germane to the claims and conclusions of an article, MCP at the outset considered that these spectra needed to be submitted with the article and be a part of the journal. Electronic publishing of biological journals, which began with the Journal of Biological Chemistry in 1995 (7), made this possible, at least at first. However, when very large-scale identifications of PTMs became commonplace, the problem was substantially exacerbated, and it became clear that, at least in some cases, it was impractical to insist that these spectra be submitted as supplemental material. In addition, many authors were meeting this compliance requirement through the production of one large pdf file of static screenshots of spectra, a format that made finding relevant spectra and more than cursory examination of the assignment difficult and reanalysis of data impossible. At the same time, the journal was also wrestling with the issue of raw data and whether it should require authors to also make this information available. It is germane to note that these were two separate issues: The spectra were considered to be a part of the manuscript, while the raw data were not. However, they were certainly interrelated. Thus, as a compromise, and in recognition of the importance of supporting repositories that store data in a secure form while allowing full public access, MCP waived the requirement that spectra be submitted with the article, allowing deposition in an acceptable public repository in lieu of inclusion in the manuscript as an alternative. At the same time, it announced that it would require all raw data to be deposited in a similar public site. Unfortunately this led, in many cases, to the often incorrect conclusion that simply depositing the raw data met both requirements without appreciating the different goals of the two submissions: one providing the ability to easily assess the authors’ interpretation of their data, the other allowing reanalysis and reuse of acquired data. This well-meaning decision to require the deposition of raw data subsequently hit “a serious snag” when one of the major sites available at the time began Received June 3, 2015, and in revised form, June 3, 2015 Published, MCP Papers in Press, June 3, 2015, DOI 10.1074/ mcp.E115.052506 Author’s Choice—Final version free via Creative Commons CC-BY license. Author contributions: A.L.B., S.A.C., R.A.B., and R.J.C. wrote the paper. Editorial